3 research outputs found
Neurosymbolic Grounding for Compositional World Models
We introduce Cosmos, a framework for object-centric world modeling that is
designed for compositional generalization (CG), i.e., high performance on
unseen input scenes obtained through the composition of known visual "atoms."
The central insight behind Cosmos is the use of a novel form of neurosymbolic
grounding. Specifically, the framework introduces two new tools: (i)
neurosymbolic scene encodings, which represent each entity in a scene using a
real vector computed using a neural encoder, as well as a vector of composable
symbols describing attributes of the entity, and (ii) a neurosymbolic attention
mechanism that binds these entities to learned rules of interaction. Cosmos is
end-to-end differentiable; also, unlike traditional neurosymbolic methods that
require representations to be manually mapped to symbols, it computes an
entity's symbolic attributes using vision-language foundation models. Through
an evaluation that considers two different forms of CG on an established
blocks-pushing domain, we show that the framework establishes a new
state-of-the-art for CG in world modeling
Neurosymbolic Programming for Science
Neurosymbolic Programming (NP) techniques have the potential to accelerate
scientific discovery. These models combine neural and symbolic components to
learn complex patterns and representations from data, using high-level concepts
or known constraints. NP techniques can interface with symbolic domain
knowledge from scientists, such as prior knowledge and experimental context, to
produce interpretable outputs. We identify opportunities and challenges between
current NP models and scientific workflows, with real-world examples from
behavior analysis in science: to enable the use of NP broadly for workflows
across the natural and social sciences.Comment: Neural Information Processing Systems 2022 - AI for science worksho
Neurosymbolic Programming for Science
Neurosymbolic Programming (NP) techniques have the potential to accelerate scientific discovery across fields. These models combine neural and symbolic components to learn complex patterns and representations from data, using high-level concepts or known constraints. As a result, NP techniques can interface with symbolic domain knowledge from scientists, such as prior knowledge and experimental context, to produce interpretable outputs. Here, we identify opportunities and challenges between current NP models and scientific workflows, with real-world examples from behavior analysis in science. We define concrete next steps to move the NP for science field forward, to enable its use broadly for workflows across the natural and social sciences.This project was supported by the the National Science Foundation under Grant No. 1918839 "Understanding the World Through Code" http://www.neurosymbolic.or